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Abstract 

In this note, we consider the higher-order Markov Chain, and characterize the second order 
Markov chains admitting every probability distribution vector as a stationary vector. The 
result is used to construct Markov chains of higher-order with the same property. We also study 
conditions under which the set of stationary vectors of the Markov chain has a certain afhne 
dimension. 
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1 Introduction 

A discrete-time Markov chain is a stochastic process with a sequence of random variables 

{AV = 0,1,2...}, 
which takes on values in a discrete finite state space 

(n) = {l,...,n} 

for a positive integer n, such that with time independent probability 

Pij = PT(X t+ i=i\Xt = j,X t -i=i t - 1 ,X t -2 = it-2,...,Xi=ii,X =i ) 
= Pv{X t+1 =i\X t =j) 

holds for all ■ ■ ■ ,it-i- The nonnegative matrix P = (pij)i<i t j< n is the transition matrix of 

the Markov process is column stochastic, i.e., J2i=iPij = 1 for j = 1, . . . , n. Denote by 

= jx = (xi, . . . ,x n y : xi, . . . ,x n > 0, X^i = 1 j (!) 

the simplex of probability vectors in R n . A nonnegative vector x G $7 n is a stationary probability 
vector (also known as the distribution) of a finite Markov Chain if Px = x. By the Perron- Frobenius 
Theory (e.g., see [3J, [8] ) every discrete-time Markov Chain has a stationary probability vector, and 
the vector is unique if the transition matrix is primitive, i.e., there is a positive integer r such that 
all entries of P r are positive. The uniqueness condition is useful when one uses numerical schemes 
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to determine the stationary vectors. With the uniqueness condition, any convergent scheme would 
lead to the unique stationary vector; e.g., see [I]. 

More generally, one may consider an m-th order Markov chain such that 

Pi,h,-,im = Pr(Xt + i = i\X t = ii,X t -i=i2,...,X 1 =it,X Q = i t+1 ) 
= Pr (Xt+i = i\Xt = «i, • • • , Xt-m+i = i m ) , 

where i,h,- ■ ■ ,i m G (n); see [HE]. In other words, the current state of the process depends on m 
past states. Observe that 

n 

y^ J Pi,i 1 ,...,i m = 1, 1 < k, ■ ■ ■ ,i m < n. 
i=i 

When m=l, it is just the standard Markov Chain. There are many situations that one would use 
the Markov Chain models. We refer readers to the papers [H El [5j El [7] and the references therein. 
Note that P = {piii,...i m ) is an ( m + l)-fold tensor of R n governing the transition of states in the 
m-th order Markov chain according to the following rule 

Xi(t + 1)= Pi,h,-,i m Xi 1 (t)--x im (t), i = l,...,n. 

l<ii,...,i m <n 

We will call P the transition probability tensor of the Markov chain. A nonnegative vector x = 
(xi, . . . , x n ) £ R ra with entries summing up to 1 is a stationary (probability distribution) vector if 

Xi = ^ ^ Ph,—,im X i\ ' ' ' X im1 i = ■ ■ ■ ,n. (2) 

Ki 

In [5] the authors used a fixed point theorem to show that a stationary vector for a higher-order 
Markov chain always exists; the stationary vector will have positive entries if the transition tensor 
P = (Pi,ii,-,i m ) 1S irreducible, i.e., there is no non-empty proper index subset I C {1,2,- •• ,n} 
such that Pi ) i 1) ... ; i m = for all i £ I, and i\, . . . ,i m ^ /. Moreover, they derived some sufficient 
conditions for the stationary vector to be unique, and proposed some iterative methods to find the 
stationary vector. As mentioned before, the uniqueness of the stationary vector is important in 
using iterative methods to find stationary vectors. 

In this note, we consider an extreme situation of the problem, namely, every probability vector 
in the simplex f2 n is a stationary vector of a higher-order Markov chain. In the standard (first- 
order) Markov chain, this can happen if and only if P is the identity matrix. We show that such 
a phenomenon may occur for a large family of higher-ordered Markov chains. In particular, we 
characterize those second order Markov chains with this property. The result is used to study 
higher-order Markov chains with the same property. 

In our discussion, we always let 

£ = {ei, . . . ,e n } 

denote the standard basis for R n . Then Q n is the convex hull of the set £. For any k £ {1, . . . , n}, 
a subset of £l n obtained by taking the convex hull of k vectors from the set £ is a face of the simplex 
il n of affine dimension k — 1. We also consider higher-order Markov chains with a (k — l)-dimension 
face of il n as the set of stationary vectors. Other geometrical features and problems concerning the 
set of stationary vectors of higher-order Markov chains will also be mentioned. 
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2 Second Order Markov Chains 



In the following, we characterize those second order Markov chains so that every vector in £l n is 
a stationary vector. Note that for a second order Markov chains the conditions for the stationary 
vector x = (xi, . . . , x n ) 1 in §Z§ can be rewritten as 



where for i = 1, . . . , n, 



x = (x\P\ H x n P n )x, 



Pi — {.Pris)l<r,s<n 



(3) 



(4) 



is a column stochastic matrix, i.e., a nonnegative matrix so that the sum of entries of each column 
is 1. We have the following theorem. 

Theorem 2.1 Suppose P = (pa 1 i 2 ) is the transition tensor of a second order Markov chain. 
Then every vector in the set Q n is a stationary vector if and only if there are nonnegative vectors 
vi, . . . , v n G R n with entries in [0, 1] such that for i = 1, . . . , n, Vj 
and 



Vn,.. - fVinY with vu = 0, 



/ 1 - Vn 



diag (v. 



a, 



, v in ) + e i w i 



\ 



f'll 



1 - 

Vi,i-1 1 

1 — Wt,*+1 



V 



1 - V in J 



To prove Theorem 12.11 we need the following detailed analysis for the second order Markov 
chain when n = 2. 

Proposition 2.2 Let a\, a,2, b%, hi G [0,1]. Consider the following equation with unknown x G [0,1]: 



+ (l-x) 



a 2 



x 

1 — x 



X 

1 — X 



1 — a% 1 — b\ J I 1 - fl2 1 — ^2 

Then one of the following holds for the above equation. 

(1) If a\ = 1, a2 + b\ = 1, &2 = 0, i/ien every x G [0, 1] is a solution. 

(2) If a,2 + b\ < 1 = a\, then there are two solutions in [0, 1], namely, x = 1 and x = 6 b ■ 

fSj // a2 + 6i — ai > a2 + 6i — 1 > = &2; i/ien i/iere are too solutions in [0, 1], namely, x = 
andx= "a+fi- 1 . 

a2+Dl-ai 

{^J Otherwise, there is a unique solution in [0,1] determined as follows. 
If a 1 -a 2 -b 1 + b 2 = 0, then x = 2 b 2 +i-a 2 -b 1 ■ 
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If a\ — a 2 — b\ + 62 7^ ; then 

with A = (26 2 + 1-02- h) 2 
Proof. Let 

/O) = ( a i 

be the first entry of the vector 

ai 61 
1 — ai 1 — bi 



26 2 + 1 - a 2 



2 (ai - 02 — 61 + 62) 
46 2 (ai - «2 - h + 6 2 ) = (1 



a 2 -6i) 2 + 46 2 (l-ai) > 0. 



a 2 - 61 + 62) x 2 + (a 2 + 61 - 26 2 ) x + b 2 



+ (1 



«2 

1 - a 2 



b 2 
I-62 



x 

1 — x 



We need only solve / (x) = x with x £ [0, 1]. Then the equation corresponding to the second entry 
will also satisfy. Set 

g(x) = f(x) - x = (01 — 02 — 61 + 62) ^ 2 + (02 + 61 - 26 2 - 1) x + 6 2 = 0. 

Then g (0) = b 2 > and 5 (1) = ai — 1 < 0. By the Intermediate Value Theorem, there is at least 
one xo G [0, 1] such that g (xq) = 0. Let 

A = (26 2 + 1 - a 2 - h) 2 - Ab 2 {a x -a 2 -b 1 + b 2 ) = (1 - a 2 - 61) 2 + 46 2 (1 - «i) > 0. 

Suppose 01 — 02 — 61+62 = 0. The quadratic equation reduces to (a 2 + 61 — 262 — 1) x+62 = 0. If 
a 2 + bi — 2b 2 — 1 = 0, then one can readily check that condition (1) holds. If a 2 + b\ — 2b 2 — 1 7^ 0, 
then the first case of condition (4) holds. 

Suppose a\ — a 2 — 61 + 62 7^ 0. If g(Q) > and g(l) < 0, then the quadratic function g(x) can 
only have one solution in [0, 1]. If a\ — a 2 — b\ + 62 > 0, then g{x) — > 00 as x — > 00. Since g(l) < 0, 
the larger root of g{x) = equals 2b 2 ^Zal-bl+i^ wn ^ ^ e l ar g er than 1. Hence, the second case of 
condition (4) holds. If 01—02 — 61 + 62 < 0, then g{x) — > —00 as x — > —00. Since g(0) > 0, the 
smaller root of g(x) = equals 2b 2 ^Za2~bi+b^ wn ^ ^ e smaner than 0. Hence, the second case of 
condition (4) holds. 

Suppose = g(l) = a% — 1. Then g(x) will have another solution in [0, 1] if and only if 
o>\ — a 2 — 61 + 62 = 1 — 02 — 61 + 62 > 0. This happens if and only if condition (2) holds. 

Suppose = g(0) = b 2 and 7^ 5(1)- Then g{x) will have another solution in [0, 1] if and only 
if ai — 02 — 61 + 62 = ai — 02 — 61 < and the maximum of g{x) is attained at a positive number 
x. This happens if and only if condition (3) holds. □ 

Proof of Theorem 12. li The sufficiency can be readily checked. We focus on the necessity. 
Note that Proposition 12.21 covers the case when n = 2. We will use an inductive argument. It is 
illustrative to see the case when n = 3. Consider the system 



On Oi2 Oi3 
O21 022 023 
031 a 32 033 



+ X 2 



hi 612 613 

b 2 i 622 623 

£>31 &32 &33 



+ Xi 



Cll C12 C13 
C21 C 2 2 C 2 3 
C31 C 32 C33 



X 



X. 
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If we set the third entry of the stationary vector x to be 0, then we can have infinitely many solu- 
te 

tions of the form x = I 1 — x I with x G [0, 1]. By the 2-by-2 case, this happens if and only if the 




i , • | "ii " j j i , / bn b\2 \ f , f (I a\2 \ ( 1 — au 

sub-matrices I I and . . are of the form „ , , 

V »21 »22 / V 1 _ a i2 / V °12 1 

Similarly, setting the second entry of x to be 0, we see that the submatrices ( 11 ° 13 ) and 

V °3i a 33 / 

c n c i3 \ are Q £ ^ e £ orm ( °13 | ( 0,13 | . Finally, setting the first entry 

c 3 i c 33 y V o i - ai3 y V °i3 1 / 

of x to be 0, we see that the sub-matrices ( ° 22 ° 23 ) and [ ° 22 C23 ] are of the form 

V fl 32 a 33 ) \ c 32 c 33 J 

. . ° 23 1,1 fl23 ? I • Thus, the three matrices in the equation are of the form 
1 - a 23 J \ a 23 1 J 

1 a 12 a i3 \ / 1 - ai 2 \ / 1 - a 13 
l-oi2 , ai2 1 a 23 , 1 - a 23 

1 - oi3 / \ 1 - a 23 / V ai 3 a 23 1 

More generally, suppose the result holds for the (n — l)-dimension case. Consider the n- 
dimension case, and the equation 

(x\P\ + • • • + x n P n )x = x with x G Q n . 

Let j £ {1, . . . , n}. Setting the j-th entry of x = (xi, . . . , x n y to be zero, we see that for i / j, the 
(n — 1) sub-matrix of Pj obtained by deleting its jth row and jth column has the form 

In-l — diag (a^i, . . . , a^j-x, Oij+i, • • • , «j,n) + ^(a^i, . . . , djj-i, Otj+i, • • • , aj jn ), 

where is obtained from e, by removing the jth entry for i = 1, . . . , n. Combining the information 
for different j = 1, . . . , n, and i = 1, . . . , j — 1, j + 1, . . . , n, we see that the matrices Pi, • • • , P n have 
the asserted form. □ 

Theorem 12.11 shows that it is possible for a second order Markov chain to have many stationary 
vectors. In |5|, the authors obtained some sufficient conditions for a higher-order Markov chain to 
have a unique stationary vector. Here we construct a family of examples of second-order Markov 
chains such that one of the following holds. 

(a) There are exactly k stationary vectors for a given k G {1, . . . , n + 1}. 

(b) The set of stationary vectors is a k dimensional face of Q n for k = 1, . . . , n — 2. 

(c) The set of stationary vectors is a disconnected set equal to the union of a A; dimensional face 
of O n and {(X)j=i e j)/ n }-> for /c = 1, . . . , n — 2. 

Theorem 2.3 Suppose n > 2 and a second order Markov chain with transition tensor P = (pi,^^)- 

Let Pi = (pris)i<r,s<n for i = 1, . . . , n. Let k G {1, . . . , n} and f k = (ex + h e k )/k. If every 

column of Pi equals fk, then fk is the only stationary vector of the Markov chain. 
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(1) If k = 2, replace the first column of Pi by e\ and all the columns of Pi by Then the 
resulting Markov chain has 2 stationary vectors, namely, e\ and e^. 

(2) If 2 < k < n, replace the ith column of Pi by and all other columns by e k for i = 1, . . . , k. 
Then the resulting Markov chain has k stationary vectors, namely, ei,... ,e&. 

(3) Suppose k = n. If we replace the ith column of Pi by and alii = 1, . . . ,n, then the resulting 
Markov chain has n + 1 stationary vectors, namely, ex, . . . , e n and f n . 

(4) If k G {2, . . . , n — 1} and we replace the first k columns of Pi by e,{ for i = 1, . . . , k, then the 
set of stationary vectors for the Markov chain equals the convex hull of {e\, . . . , e k }- 

(5) Suppose k G {2, . . . ,n — 1} and we reset the matrices P±, . . . ,P n so that the first k columns 
of Pi equal 



((efc+H Ve n )/{n-k) if i = k+1, . . . , n, 

and all other columns equal to f n . Then the set of stationary vectors for the Markov chain 
equals the union of the convex hull of {e±, . . . , e^} the set {fn}- 

Proof. Suppose k G {1, . . . , n} and every column of Pi equals fk- Then x = [x\, . . . , G £l n 
satisfies 

x = {xiPi H h x n P n )x = (xi H h x k )f k 

if and only if x^+i = • • • = x n = and x% = ■ ■ ■ = Xf. = 1/k. 

(1) Suppose k = 2, and we replace Pi and P2 as suggested. Then x G Q n satisfies 

x = {x\P\ H h x n P n )x 

if and only if X3 = ■ ■ ■ = x n = and 



By Proposition 12. 2\ x\ = 1 or x% = 1. So, the Markov chain has two stationary vectors e\ and e2- 

(2) Suppose k > 2, and the ith column of Pi is replaced by ej and replace all other columns 
by e k for i = 1, . . . , k. Direct checking shows that e±, . . . , e k and fk are stationary vectors of the 
Markov chain. Conversely, suppose x = (xi, . . . , x n y G O n satisfies 




if i 



1, ...,k, 




x = 



(xiPi H h x n P n )x. 



Then Xk+i = ■ ■ ■ = x. 



= 



fc-i 





= xj for j = 1, . . . , k — 1. 
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Thus, Xj G {0, 1} so that x = ej if Xj = 1 for any j = 1, . . . , k — 1. If Xi = ■ ■ ■ = Xfc_i = 0, then Xk 
is the only nonzero entry and x = e k . 

(3) Suppose k = n and we replace the ith column of Pi by ej and alH = 1, . . . , n. Direct computa- 
tion shows that ei, . . . , e„ and f n are stationary vectors. Conversely, suppose x = (xi, . . . , x n )' G Q n 
satisfies 

x = (xiPi H h x„P n )x. 

Then 

Xl = M E ^Xj-Y^x) \+x} = - 1-E X ? 

\l<ij<n i=l / \ j=l / 

Let £ = ^ ^1 — Ylj=i x j) an d consider two cases. 

Case 1. If £ = 0, then Xj G {0, 1} for each i = 1, . . . , n. Thus, we have x G {ei, . . . , e n }. 

Case 2. Suppose £ > 0. Because xf — Xi + 1 = 0, we see that Xj = (l ± a/1 — 4/) /2. If at least 
one of the Xj's equals (l + y/1 — 4l) /2, then by the fact that n > 2, 

l = E*i> 2 + ( n - !)" ~ 2 = 1 + - 2) ^ ^ ^ > 1, 

j'=i 



which is a contradiction. Thus, Xj = (l — a/1 — 4/) /2 for each z = 1, . . . , n, and hence x = f n . 

(4) Clearly, every vector in the convex hull of {ei, . . . , e k } is a stationary vector of the Markov 
chain. Conversely, suppose x = (xi, . . . ,x n )' G fl n is a stationary vector. Then 

x = (xiPi H h x„P n )x 

implies that Xk+i = • • • = x n = 0, and x±, . . . , Xk can be any nonnegative numbers usmming up to 
one. 

(5) One readily checks that f n and every vector in the convex hull of {ei, . . . , e k } is a stationary 
vector of the Markov chain. Conversely, suppose x = (xi, . . . , x n )* G £l n is a stationary vector. If 

I 3 = (Y!j=k+i x i)i then 

x = (xiPi H h x n P„)x 

= (1 - /3)(xiei + • • • + x k e k ) + /3/ n + (1 - P)0(e k+1 + ■■■ + e n )/(n - k). 

Thus, Xfc+i = • • • = x n . If all of them are zero, then be any nonnegative numbers 

summing up to one. If x k +i = • • • = x n = r > 0, then Xj = (1 — /3)xj + j3/n so that Xj = 1/n for 
i = 1, . . . , k. If follows that x k +\ = ■ ■ ■ = x n = r = 1/n also. □ 

Next, we obtain a result illustrating some additional geometrical feature of the set of stationary 
vectors of a second order Markov chain. 
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Proposition 2.4 Consider the following equation for the stationary vectors of a second order 
Markov chain: 

[x\P\ H h x n P n )x = x. 

Suppose the Markov chain has two stationary vector of the form xei + (1 — x)ej and yei + (1 — y)ej 
for some x,y £ (0, 1) and 1 < i < j < n. Then every vector of the form ze{ + (1 — z)ej with 
z £ [0, 1] is a stationary vector of the Markov chain. 

Proof. If n = 2, the result follows from Proposition 12.21 Suppose n > 3. The hypothesis of the 
proposition implies that the 2-by-2 submatrices of Pi and Pj lying in rows and columns indexed by 

i and j have the form ( „ _ a | and | ° ? V It follows that every vector of the form 

\ 1-a J \ a 1 J 

zei + (1 — z)ej with z £ [0, 1] is a stationary vector of the Markov chain. □ 

Proposition 12.41 asserts that if the set of stationary vectors of a second order Markov chain 
contains two interior points of a 1-dimensional face of O n , then every vector in the 1-dimensional 
face is a stationary vector. 

We conjecture that if the set of stationary vectors of a second order Markov chain contains k 
interior points of a (k — 1) dimensional face of the simplex Q m then every vector in the (k — 1) 
dimensional face is a stationary vector. 

3 Higher- Order Markov Chains 

In this section, we use the results in Section 2 to construct higher-order Markov chains so that 

(I) every vector in £l n is a stationary vector, and 

(II) the set of stationary vectors have different affine dimensions. 

We will identify a transition probability tensor P = {pa x ... j m ) as an n x n m matrix with row index 
i = 1, . . . , n, and column indexes i\ ■ ■ ■ i m with i%, . . . , i n £ (n) = {1, . . . , n} arranged in lexico- 
graphic order. For example, for n = 2 and m = 3, the row indexes are 1, 2, and the column indexes 
are 111,112,121,122,211,212,221,222. We will use the tensor (Kronecker) product notation for 
vectors in Q, n . For example, 

X 1 - 3 ) = X (8) X ® X = (x\XiX\, X\X\X2, X\X2X\,X\X2X2,XiX\X\,X2X\X2^ X2X2X1 , X2X2X2Y ■ 

The the stationary vector condition can be represented as the following matrix equation: 

Px( m ) = x. 

We first consider Markov chains satisfying condition (I). We will illustrate the construction for 
the third order Markov chains for n = 2, and then describe the general construction. 

First order Markov chains. Every vector in fi n is a stationary vector if and only if P = I2. 
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Second order Markov chains. We can use the construction for the first order chain P = J 2 to 
produce the following. 

[P\P]x=( Plu Pl12 Pl21 Pl22 )x^=x withx=(x 1 x 1 ,s 1 x 2 ,x 2 x 1 ,x 2 x 2 )*. 

VP211 P212 P221 P222/ 

So, Q = [P\P] = [^2^2] satisfies Qx^ = x for every x G f2. Observe that the second and 
third entries on x^ 2 ) are the same, so one can permute the second the third columns of [P, P] to 
get Q = (^q q ^ ^ such that Qx^ = x for every x G 12. Evidently, for every a G [0,1], 

Qa = a Q + (1 — o)Q will satisfy <3 a x( 2 ) = x. In fact, we have shown that these are all possible 
transition tensors have the desired property. 

Third order Markov chains. Suppose Q = ( ^ m ^ 112 ^ 121 ^ 122 ) satisfies Qx^ 2 ) = x for 

\Q211 9212 9221 9222/ 

every x G O n . If R = [Q|Q], then Rx^ = = x for every x G Q n . Now, observe that the entries 
of x^ 3 ) indexed by 112, 121, 211 are all equal to x 2 x 2 . So, we can permute the columns of R indexed 
by 112, 121, 211 in 3! = 6 different ways to get a matrix with the desired properties. Similarly, we 
can permute the columns of R indexed by 122, 212, 221 to get different matrices with the desired 
properties. As a result, we get 6 2 = 36 different matrices with the desired property. Now, we can 
take convex combination these matrices to get a large family of matrices with the desired property. 

One easily extends the above idea to obtain the following. 

Theorem 3.1 Suppose m,n > 2, and P is a transition probability tensor P represented as an 
n x n m matrix such that Px m = x for all x G £l n . Let Q = [P\ ■ ■ ■ \P] = 1 (g) P with 1 = (1, . . . , 1) 
with m copies. Then 

Q x (^+i) = (l(g)P)( x( g)x( m )) = (lx)®(Px( m ))=x forallx£Q n . (5) 

Moreover, one can permute the columns of Q corresponding to the entries in the vector x( m+1 ) with 
the same values: x™ 1 ■ ■ ■ x™ n for all nonnegative sequence (mi, . . . , m n ) with mi + • • • + m n = m to 
yield other Markov chains satisfying f5|); in addition, taking convex combination of these matrices 
will also result in Markov chains satisfying {5|j. 

Note that there are 

m \ ml 



v mi,m 2 , • • • ,m r J mi!---m n ! 
so many terms in the vector x^" 1 ) , and hence there are ( m ^ m ) ! permutations for the corre- 
sponding columns in Q. Thus, we can generate many new matrices Q from Q satisfying Qx^ m+1 ^ = x 
for all x G f2 n . 

An interesting question is whether a higher-order Markov chain with transition tensor Q satis- 
fying Qx^ = x for every x G Sl n can be obtained from the above construction. 

Next, we turn to higher-order Markov chains satisfying condition (II). 



9 



Theorem 3.2 Suppose n > 2 and an m-th order Markov chain with transition tensor P = 
(pi,h,...i m ) identified as and n x n m matrix. Partition P = [Pi\ ■ ■ ■ \P n ] so that the Pi is n x n m ~ l . 
Let k £ {1, . . . , n} and = (ei + • • • + e^)jk. If every column of Pi equals fk, then f^ is the only 
stationary vector of the Markov chain. 

(1) If k = 2, replace the first volume of P\ by e\ and replace all the columns of P2 by ei- Then 
the resulting Markov chain has 2 stationary vectors, namely, e\ and ei- 

(2) If 2 < k < n, replace the column of Pi indexed by (ii,...,i m ) = (i,...,i) by ei for all 
i = 1, . . . , k, and all other columns by for i = 1, . . . , k, then the resulting Markov chain has 
k stationary vectors, namely, ei, . . . ,e&. 

(3) Suppose k = n. If we replace the the column of Pi indexed by . . . , i m ) = (i, . . . , i) for all 
i = 1, . . . , n, then the resulting Markov chain has n + 1 stationary vectors, namely, e%, . . . , e n 
and f n . 

(4) If k £ {2, ...,n— 1} and we replace the columns of Pi indexed by (ii,...,i m ) with 1 < 
iii ■ ■ ■ j *m ^ by ei for i = 1, . . . , k, then the set of stationary vectors for the Markov chain 
equals the convex hull of {e±, . . . , e^}. 

(5) Suppose k G {2, . . . , n — 1} and we reset the matrices P\, . . . , P n so that for each i = 1, . . . ,n, 
the columns of Pi indexed by . . . , i m ) with 1 < ii, . . . , i m < k, equal 



[ (e k+ x H h e n )/(n - k) if i = k+1, n, 

and all other columns of Pi equal f n . Then the set of stationary vectors for the Markov chain 
equals the union of the convex hull of {e\, . . . , e^} the set {fn}- 



To conclude our note, we remark that there are many interesting questions concerning the 
stationary vectors of higher-order Markov chains that deserve further study. 
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Proof. The proof is an easy adaptation of that of Theorem 12.31 
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